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Abstract — In order to understand the performance of a code 
under maximum-likelihood (ML) decoding, it is crucial to know 
the minimal codewords. In the context of linear programming 
(LP) decoding, it turns out to be necessary to know the minimal 
pseudo-codewords. This paper studies the minimal codewords 
and minimal pseudo-codewords of some families of codes derived 
from projective and Euclidean planes. Although our numerical 
results are only for codes of very modest length, they suggest that 
these code families exhibit an interesting property. Namely, all 
minimal pseudo-codewords that are not multiples of a minimal 
codeword have an AWGNC pseudo-weight that is strictly larger 
than the minimum Hamming weight of the code. This observation 
has positive consequences not only for LP decoding but also for 
iterative decoding. 

I. Introduction 

Our motivation for looking at minimal codewords and 
minimal pseudo-codewords (PCWs) is twofold. On the one 
hand we would like to be able to give performance guarantees 
of the LP decoder, on the other hand, the connection made 
by Koetter and Vontobel [1], [2] between iterative decoding 
and LP decoding suggests that results for LP decoding have 
immediate implications for iterative decoding. In this paper we 
focus solely on certain families of codes based on projective 
and Euclidean planes. One of the reasons why these families 
are worthwhile study objects is that in the past, several groups 
of authors have experimentally observed that codes from these 
families can perform very well under iterative decoding, see 
e.g. [3], [4]. Another reason is that these families of codes have 
concise descriptions and large automorphism groups which 
may potentially be used to simplify their analysis. 

More precisely, the codes under investigation are the fam- 
ilies of codes that were called type-I PG-LDPC and type-I 
EG-LDPC codes in [4]. Type-I PG-LDPC codes are defined 
as follows. Let q = 2^ for some positive integer s and consider 
a (finite) projective plane PG(2, q) (see e.g. [5]) with q'^+q+1 
points and q"^ + q+1 lines: each point lies on g + 1 lines and 
each line contains q+1 points.' A standard way of associating 
a parity-check matrix H of a binary linear code to a finite 
geometry is to let the columns of H correspond to the set of 
points, to let the rows of H correspond to the set of lines, and 
finally to define the entries of H according to the incidence 
structure of the finite geometry. In this way, we can associate 
to the projective plane PG(2, q) the code CpG(2,g) with parity- 
check matrix H = llpG{2,q) whose parameters are: 

'Note that the "2" in PG(2,g) stands for the dimensionahty of the 
geometry, which in the case of planes is 2. 



length 
dimension 

minimum Hamming distance 
uniform column weight of H 
uniform row weight of H 
size of H 



= 9 + 9 + 1 
fc = ?i - 3"' - 1 

rfmin =9 + 2 
Wcol = 9+1 
Wrow =9+1 

n X 71 



Type-I EG-LDPC codes are defined as follows. Let 9 = 2" 
for some positive integer s and consider a (finite) Euclidean 
plane EG(2, q) (see e.g. [5]) with q^ points and q"^ + q lines: 
each point lies on 9 + 1 lines and each line contains 9 points. 
We essentially use the same procedure as outlined above in 
order to associate a parity-check matrix to a finite geometry. 
But before doing this, we modify the Euclidean plane slightly: 
we select a point of EG(2, 9) and remove it together with the 
9+1 lines through it. Doing so, we obtain an EG(2, 9)-based 
code CEG(2.q) with parity-check matrix H = Heg(2,(}) whose 
parameters are: 



length 
dimension 

minimum Hamming distance 
uniform column weight of H 
uniform row weight of H 
size of H 



n 

k : 

dn 

Wr 



= 9 - 
: n — 3 

n = 9- 
1 = 9 

W^row — 9 

n X n 



1 



1 



Both families of codes have the nice property that with 
an appropriate ordering of the columns and rows, the parity- 
check matrix is a circulant matrix, meaning that CpG(2,(j) 
and CEG(2,q) cyclic codes. This fact can e.g. be used 
for efficient encoding. Such symmetries can also substantially 
simplify the analysis; let us point out that the automorphism 
groups of CpG(2_5) and CEG(2,g) actually contain many more 
automorphisms besides the cyclic-shift automorphism implied 
by the cyclicity of the codes. 

The structure of the rest of the paper is as follows. Sec. |ll| 
discusses ML and LP decoding and Sees. IIIII and II VI introduce 
minimal codewords and minimal PCWs, respectively. The aim 
of these earlier sections is to set the stage for Sec. [V] and to 
enable the reader to appreciate the numerical results presented 
therein for certain selected codes. Finally, in Sec. lVIl we state 
some concluding remarks. 

II. ML AND LP Decoding 

In this section we briefly review ML and LP decoding. 
Consider a binary linear code C of length n and dimension k 



that is used for data transmission over a memoryless binary- 
input channel. The codeword that is transmitted will be called 
X whereas the received vector will be called y. Based on the 
received vector, we can define the log-likelihood ratios (LLRs) 

to be A, = log(pY;|x.(ydO)/Py.|x.(y«|l))' « = 1, ML 
decoding can then be cast as 

n 

X = argminy^ XiAi, (1) 

i=l 

Letting conv(C) be the convex hull of C in M", the above ML 
decoding rule can also be formulated as 

n 

X = arg min 2,^i^i- (2) 

xGconv(C) — ^ 
i—1 

Unfortunately, for most codes of interest, the description 
complexity of conv(C) grows exponentially in the block 
length and therefore finding the minimum in Q with a linear 
programming solver is highly impractical for reasonably long 
codes.^ 

The next step is to use a standard approach from optimiza- 
tion practice: we replace the minimization over conv(C) by a 
minimization over some easily describable polytope V that is 
a relaxation of conv(C): 

n 

x^argmin> XiXi. (3) 

i—l 

If V is strictly larger than conv(C) then the decision rule 
in (|3 obviously represents a sub-optimal decoder A relaxation 
that works particularly well for LDPC codes is given by 
the following approach [6], [7]. Let C be described by an 
mxn parity-check matrix H with rows hi, h2, . . . , h,„. Then 
the polytope V = ^(H), in this context also called the 
fundamental polytope [1], is defined as 

m 

V =f] conv(Ci) with Ci = |x e {0, 1}" I h,x"^ = mod 2 1 . 

1=1 

Note that P is a convex set within [0, 1]" that contains conv(C) 
but whose description complexity is much smaller than the 
one of conv(C). Points in the set V will be called PCWs. 
Because the set V is usually strictly larger than conv(C), it 
can obviously happen that the decoding rule in (|3} delivers 
a vertex of V that is not a codeword. Such vertices are the 
reason for the sub-optimality of LP decoding (cf. [1], [7]). 
Note that P(H) is a function of the parity-check matrix H 
that describes the code C; different parity-check matrices for 
the same code might therefore lead to different fundamental 
poly topes. 

in. Minimal Codewords 

Although ML decoding is often impractical, knowing 
bounds on the block error rate of an ML decoder can help 
in assessing the performance of sub-optimal but practical 
decoding algorithms. 

Definition 1 (cf. e.g. [8], [9]): Let C be a binary linear 
code of length n. For x G C, let V^^ ^ { A e M" | x' • > 

^Exceptions to this observation include for example the class of convolu- 
tional codes with not too many states. 



X • A^ for all x' G C \ {x} } be the region in the LLR space 
where the ML decoder decides in favor of the codeword x.^ 

□ 

In the following, we will assume that we use a binary 
linear code C for data transmission over a binary-input output- 
symmetric channel. For this setup, when studying the ML 
decoder in Q or Q, we can without loss of generality assume 
that the zero codeword was sent, because all decision regions 
are congruent. 

Our interest in the following definition will become apparent 
in Th. |3] below. 

Definition 2: Let the support of a vector x be defined as 
supp(x) = {i I Xi 7^ O} and let C be a binary code. A non-zero 
codeword x G C is called minimal if and only if its support 
does not (strictly) contain the support of any other non-zero 
codeword as a proper subset. The set of all minimal codewords 
of C will be denoted by M{C). □ 

Theorem 3 (cf. e.g. [9]): Let C be a binary linear code of 
length n. The decision region of a codeword x G C 

shares a facet with the decision region T>^^ of the zero 
codeword if and only if x G A4(C). □ 

Therefore, knowing the minimal codewords of the code C 
is sufficient in order to assess its ML decoding performance. 
(Further results about minimal codewords can e.g. be found 
in [8H11].) 

IV. The Fundamental Cone and 
Minimal Pseudo-Codewords 

For LP decoding of a binary linear code that is used for data 
transmission over a binary-input output-symmetric channel, it 
is sufficient to consider the part of the fundamental polytope 
V around the vertex 0, cf. [1]. (See also [6], [7] that discuss 
this so-called "C-symmetry" property.) 

Definition 4 ([1], [7]): Let C be an arbitrary binary linear 
code and let H be its parity-check matrix. We let J' = J^(H) 
be the set of row indices of H and we let X = X(H) be the set 
of column indices of H, respectively. For each j G J, we let 
Xj ^ Ij(H) = |i G I I /iji = l}. We define the fundamental 
cone /C(H) of H to be the set of vectors lj G M" that satisfy 

Vi G X ; > 0, 

Vj G J, Vi G Xj ; J2 

The fundamental cone defined in Def.|4]is exactly the part of 
the fundamental polytope V around the vertex and stretched 
to infinity. We note that if a; G /C(H), then also a-u} e /C(H) 
for any a > 0. Moreover, for any G /C(H) there exists an 
a > (in fact, a whole interval of a's) such that a-a; G 'P(H). 

For a given binary linear code C with parity-check matrix H, 
the importance of the set /C(H) lies in the following fact. Let 
2?LP A g Rn I ^.;^T > Q f^j. ^jj ^ ^ p(H)\{0}} be the 

region where the LP decoder decides in favor of the codeword 
0."* It can easily be seen that V^^ = {A G M" | a; • A"^ > 
for all uj G /C(H) \ {0}}. Therefore, when studying LP 

'Note that during ML decoding, ties between decoding regions can either 
be resolved in a random or in a systematic fashion. 

"'Note that during LP decoding, ties between decoding regions can either 
be resolved in a random or in a systematic fashion. 



decoding it is enough to know /C(H); all vectors lj E /C(H) 
will henceforth be called PCWs. 

Definition 5 ([1]): Let C be an arbitrary binary linear code 
described by the parity-check matrix H whose fundamental 
cone is /C(H). A vector u) e A^(H) is called a minimal PCW 
if the set {a • a; | a > 0} is an edge of /C(H). Moreover, the 
set of all minimal PCWs will be called Mp{IC{Il)).^ □ 
For a given binary linear code C with parity-check matrix H, 
the importance of the set A^p(/C(H)) lies in the following fact. 
From basic cone properties (cf. e.g. [12]), it can easily be seen 
that = {A e M" I u; • > for all u e 7Wp(/C(H))}. 
Therefore, the set A^p(/C(H)) completely characterizes the 
behavior of the LP decoder. 

Definition 6: Let C be an arbitrary binary linear code de- 
scribed by the parity-check matrix H. The additive white 
Gaussian noise channel (AWGNC) pseudo-weight [13] of 
a PCW e /C(H) is defined to be w^wgnC(^) ^ 
I'^l li/l I'^l li' where lltLijli and ||a;||2 are the Li- and L2- 
norm of a;, respectively.^ □ 
The significance of Wp^*^'^'-^(a;) is the following: it can be 
shown that the squared Euclidean distance from the point +1 
in signal space (which corresponds to the codeword 0) to the 



plane {A G | w • A"^ = 0} 



IS w 



AWGNC 



Definition 7: Let C be an arbitrary binary linear code. 
We define the codeword weight enumerator and the minimal 
codeword weight enumerator to be the polynomials 

and xf''^{X)= ^ X™h(x)^ 

xSC x6A1(C) 

respectively. □ 
Definition 8: Let C be an arbitrary binary linear code de- 
scribed by the parity-check matrix H. We define the minimal 
PCW AWGNC pseudo-weight enumerator to be the polynomial 
(with potentially non-integer exponents) 

E 

[i^]SA1p(K{H)) 



MPCW. AWGNC / v\ 



where the summation is over all equivalence classes of mini- 
mal PCWs.^ □ 
Definition 9: Let C be an arbitrary binary linear code de- 
scribed by the parity-check matrix H and let A^p(A^(H)) be 
the set of all minimal PCWs that are not multiples of minimal 
codewords. We call the real-valued quantity 



5(H) ^ 



mm 1 



the pseudo-weight spectrum gap of H. □ 
Using Cor. 8 in [1] one can show that for a randomly 
constructed (wcoi, Wrow)-regular code with 3 < Wcoi < Wrow 
the pseudo-weight spectrum gap becomes strictly negative 
with probability one as the block length goes to infinity. 
However, using Th. 1 in [14] one can show that for the 
PG(2, q)- and EG(2, g)-based codes (with square parity-check 
matrix as discussed in Sec. |l} the pseudo-weight spectrum 
gap is non-negative . In fact, we will see that for the codes 

'Note that this definition implies that ^ Mp(JC(H)). 



*We set 



"(x) = for X = 0. Note that for x G {0, 1}" we have 



tOp"^ ^(x) = uih(x), where uih(x) is the Hamming weight of x. 

'Two PCWs £ /C(H) are in the same equivalence class if there 

exists an o > such that = a ■ to' . 



investigated in Sec. [V] the pseudo-weight spectrum gap is 
significantly positive. We note that by applying simple per- 
formance bounding techniques it can be shown that the larger 
the gap is, the closer is the LP decoding performance (and 
potentially also the iterative decoding performance [1]) to the 
ML decoding performance as the SNR goes to infinity. 

Before we turn to some numerical results about minimal 
codewords and minimal PCWs, let us mention some related 
work by Kashyap and Vardy [15] which discusses results that 
characterize (minimal) stopping sets for finite-geometry-based 
codes. This is of some relevance to this paper because it is 
well-known that the support set of any PCW is a stopping 
set and that for any stopping set there exists a PCW whose 
support set equals that stopping set [1], [7]. 

V. Numerical Results 

In this section we present minimal PCWs, weight enumer- 
ators, and the pseudo-weight spectrum gap for some short 
PG(2, q)- and EG(2, g)-based codes. 

A. Type-I PG-LDPC code for q = 2 

The PG(2, 2)-based code CpG(2,2) of type I has parameters 
[n—7, k^3, dmin=4] and can be represented by the following 
circulant parity-check matrix of size 7x7: 



H 



PG(2,2) 



/I 





1 



\1 



The set Ai{C) of minimal codewords consists of the following 
codewords: 

(1,0,0,1,0,1,1), (1,1,0,0,1,0,1), (1,1,1,0,0,1,0), (0,1,1,1,0,0,1), 
(1,0,1,1,1,0,0), (0,1,0,1,1,1,0), (0,0,1,0,1,1,1). 

Obviously, all of them have Hamming weight 4 and they are 
all cyclic shifts of each other. Because the code has 2'^ = 8 
codewords in total, it turns that this code is special in the 
following sense: there are no non-zero codewords that are not 
minimal codewords. 

The set A1p(/C(H)) of minimal codewords contains all the 
elements from A4{C) plus the following PCWs that are not 
codewords (we show one representative per equivalence class): 

(1.2.2.1.2.1.1) , (1,1,2,2,1,2,1), (1,1,1,2,2,1,2), (2,1,1,1,2,2,1), 

(1.2.1.1.1.2.2) , (2,1,2,1,1,1,2), (2,2,1,2,1,1,1). 

All these minimal PCWs that are not codewords turn out to 
be cyclic shifts of each other and to have AWGNC pseudo- 
weight ^ = 6.25. The weight enumerators for this code are 
therefore: 



cw 



Xc 



PG(2,2) 
MOW 
-^CpQ(2,2) 
MP CW, AWGNC 
''^HpG(2,2) 



(X) =X" + 7X^ 

(X) = 7X\ 

(X) = 7X"* + TX'^ 



Hence, the pseudo-weight spectrum gap is ,9(HpG(2.2)) = 
6.25 -4 = 2.25. 

The codes introduced in Sec.|l]were based on square parity- 
check matrices. However, the code PG(2, 2) can also be 
described by a parity-check matrix Jip^^^ 2) of size 4x7 
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Fig. 1. Histogram of the AWGNC pseudo-weight of minimal PCWs of tlie 
PG (2, 4) -based code. (Note that tlie y-axis is logarithmic.) 



which is equivalent to the first four lines of the HpG(2.2)- The 
minimal PCWs that are not codewords turn out to be (we show 
one representative per equivalence class): 

(3,2,1,1,1,0,0), (0,1,2,1,1,3,0), (0,1,1,1,2,0,3), (0,1,1,1,1,0,0), 
(2,1,1,1,0,0,1), (2,1,0,1,1,1,0), (1,2,1,1,1,0,0), (0,1,2,1,1,1,0), 
(0,1,1,1,2,0,1), (0,1,1,1,0,2,1), (1,0,1,1,1,2,0), (1,0,1,1,1,0,2), 
(0,1,0,1,1,1,2), 

from which follows that 



MPCW, AWGNC/ 
PG(2,2) 



Note that the pseudo-weight spectrum gap is g(Hj 



4-4 = 0. Comparing the enumerator XHpoJ^'f^^^^^^) 
with the enumerator •^^pcw,awgnC/jj^\ jj. apparent that the 

PG(2,2) 

performance of LP decoding using the second representation 
will be worse than the performance of LP decoding using the 
first representation. Based on iterative decoder simulations, 
MacKay and Davey [16, Sec. 4] observed a similar perfor- 
mance hierarchy between different representations of the same 
code. (Note that the code under investigation in [16] was the 
PG(2, 16)-based code.) 

B. Type-I PG-LDPC code for q ^ A 

The parity-check matrix HpQ(2.4) of the PG(2, 4)-based 
code CpG(2.4) has size 21x21, uniform column and row weight 
5, and yields a code with parameters [7i=21, A;=ll, (i,nin=6]. 
The codeword weight enumerator and the minimal codeword 
weight enumerator are 



cw 
Xc 



:pG(2 4) (^) = ^° + + + lOOSX^ 



MOW 



Xc 



■PG(2,4) 



(X) = 168X^ + 210X^ + 1008X 



10 



respectively. Looking at these enumerators we see that all 
codewords with Hamming weight 6, 8, and 10 are also mini- 
mal codewords. Analyzing the set of all weight-6 codewords 
one sees that they all have the same pattern, i.e. they can 
all be obtained from a single weight-6 codeword by applying 
a suitable PG(2, 4)-automorphism. The same is true for all 
other sets of codewords with the same weight. This makes the 
classification of all the codewords of CpG(2,4)> and in particular 
of the minimal codewords of CpG(2.4)i relatively easy. 



Instead of giving the formula for Xh 



MPCW, AWGNC 



(X), we 



o c '^^"PG(2,4U_ ^ ' 

simply give its histogram (spectrum), cf. Fig.Q] The gap turns 
out to be .g(HpG(2,4)) = 9.8 - 6 = 3.8. 
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Fig. 2. Histogram of tlie AWGNC pseudo-weight of minimal PCWs of the 
EG(2, 4)-based code. (Note that the y-axis is logarithmic.) Top: For 15 X 15 
parity-check matrix Heg(2,4). 9(Heg(2,4)) = — 5 ~ 8.89 — 5 = 3.89. 
Middle: For 9 X 15 parity-check matrix Hj^,-,,, gCH'-^^^^ = ^ — 



5 Si 5.44 - 

5(^EG(2,4) 



5 = 0.44. Bottom: For 8 
) = ^ - 5 Si 4.69 - 5 



EG(2,4)' 

X 15 parity-check matrix HgQj2 4)' 
= -0.31. 



C. Type-I PG-LDPC code for q = 8 

Judging from some very preliminary results (based on 
random search experiments) the pseudo-weight spectrum gap 
5(HpG(2.8)) for the PG(2, 8)-based code seems to be at 
least 6.O.' 

D. Type-I EG-LDPC code for q ^ 4 



The parity-check matrix H 



EG(2,4) 



of the EG(2,4)-based 
code Ceg(2,4) has size 15x15, uniform column and row weight 
4, and yields a code with parameters [n=15, fc=7, d„iin=5]. 
The codeword weight enumerator and the minimal codeword 
weight enumerator are 

X?^(2.4, (^) = ^° + 18^' + 30X^ + 

respectively. Looking at these enumerators we see that all 
codewords with Hamming weight 5, 6, 7, 8, and 9 are 
also minimal codewords. Analyzing the set of all weight- 
5 codewords one sees that they all have the same pattern, 
i.e. they can all be obtained from a single weight-5 codeword 
by applying a suitable EG(2, 4)-automorphism. The same is 
true for all other sets of codewords with the same weight. 

The histograms (spectra) in Fig. |2 correspond to various 
parity-check matrices that describe Ceg(2.4)- Fig- IZl (top) 




* EG(2,4)-based code: SPA (after max. 8 Iterations} 

* EG(2,4)-based code: SPA (after max. 256 Iterations) 
— V — EG(2,4)-based code: LP-union upper bound 

— * — EG(2,4)-based code: LP 

-V- EG(2,4)-based code: ML-union upper bound 
EG{2,4}-based code: ML 

-A- EG(2,4}-based code: ML Seguin lower bound 
- H - EG(2,4)-based code: LP UUB minus ML SLB 



EINjaB] 



Fig. 3. Word error rate for various decoding algorithms togetlier witli some 
upper and lower bounds. (See main text for explanations.) 



VI. Concluding Remarks 

We have investigated the minimal PCWs of some simple 
PG(2, q)- and EG(2, q)-hased binary linear codes and we have 
introduced the notion of a pseudo-weight spectrum gap for a 
parity-check matrix, a concept which is certainly worthwhile 
to be further explored. Although our numerical results are for 
codes of very modest size, to the best of our knowledge this is 
the first study that tries to analytically quantify the behavior of 
PG(2, q)- and EG(2, (7)-based binary linear codes under LP 
and iterative decoding. Extending these results to somewhat 
longer codes has the potential to explain many experimental 
observations made in the past. 
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shows the histogram for XapG^ f'^'^^'^ i-^)-' Fig- 12 (middle) 
shows the histogram for -^^pcW,awgnC/jj^\ ^j^gj-g Jl' 

^PG(2,4) ^ ' ^0(2, 4) 

is a randomly selected 9 x 15 submatrix (with column weights 
at least 2) of HpQ(2 4); and finally Fig. |2] (bottom) shows 
the histogram for Xn" 



(X) where Hp„.„ is an 

PG(2,4) ^ ' ^"^(2,4) 

X 15 submatrix (with five columns having weight only one) 



of consecutive rows of the (circulant) matrix H 



PG(2,4)- 



It 



can easily be seen that for the parity-check matrices under 
investigation those with more lines lead to more favorable 
histograms. 

In Fig. |3] we show various decoding simulation results for 
data transmission over a binary-input AWGNC and lower 
and upper bounds: HEG(2,4)-based sum-product algorithm 
decoding, HEG(2.4)-based LP decoding, CEG(2.4)-based ML 
decoding, an upper bound on LP decoding based on a union 
of events upper bound, an upper bound on ML decoding based 
on a union of events upper bound, and a lower bound on ML 
decoding based on an inequality by de Caen as presented by 
Seguin [17]. It can be seen that thanks to the knowledge of 
minimal codewords and minimal PCWs we are able to obtain 
bounds that are very tight from a certain SNR value on. 



E. How the results were obtained 

Let us briefly mention how the results for the minimal 
PCWs were obtained. We used the program "Irs" [18] to search 
edges in cones. For the code CpG(2.4) we additionally used the 
two-transitivity of the points of a projective plane in order to 
formulate a simpler edge-enumeration subproblem which can 
be solved efficiently and from which all the minimal PCWs 
can be derived. There are various other ways to use the large 
automorphism groups of these codes that help in simplifying 
the edge-enumerating problem. Properties of minimal code- 
words might also be used towards that goal. 
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